1. Field of the Invention
This invention relates to information processing systems, information processing apparatuses, information processing methods, and computer programs.
2. Description of the Related Art
With the progress of today's communication technology, device technology such as cameras or projectors, and computing technology, computers, cameras, and projectors are widely spreading in working places. In combination with these technologies, communication systems and computer systems are capable of supporting various types of business activities performed at remote places. For instance, teleconferencing, videotelephony, and video chatting are frequently carried out in industrial activities and in a variety of everyday life. The network system, computer, camera, and projector installed on a meeting room can support the operation of remote teleconferencing.
Videoconferencing is a representative example of remote teleconferencing support systems. On the videoconferencing, image and sound input devices such as cameras and microphones and image and sound output devices such as monitors and speakers are installed in multiple conference rooms located at remote sites. The videoconferencing supports and promotes the communication between the remote sites.
With the videoconferencing system, various types of materials related to the conference, namely, whiteboards to be used for the conference, slides, and other presentation documents may be shared and stored in the conference rooms or personal desktop computers. For example, NetMeeting of Microsoft is known as typical desktop conferencing software. Users in the remotes sites, who are communicating over NetMeeting, are able to share conference materials of PowerPoint file and graphic data created with the drawing function.
In addition, the graphic data can be shared in real time by connecting electric blackboards or whiteboards to the network. For example, with SMARTBoard of SMART Technologies Inc., it is possible to display and share a handwritten drawing in the remote sites by utilizing a large-sized screen and a touch panel. Here, with the above-mentioned technologies, basically, only electronically-stored original data can be displayed and shared. However, while an actual conference is being held, there is a demand that the image data of a real thing existent in a three-dimensional space and the electric data are superimposed and displayed to share such superimposed image data.
There have been several proposals for the technique of superimposing the real thing (object) and the electric data for displaying such superimposed image data. For example, iLight system, having been developed by FXPAL (Fuji Xerox Palo Alto Laboratory), employs a projector and a camera and builds a server/client system. With the iLight system, users are able to display and share the graphic data at remote sites, for example, as described in Jonathan Foote and Don Kimber: “Remote Interactive Graffiti”, MM'04, Oct. 10-16, 2004 (hereinafter, simply referred to as Document 1).
Specifically, in the iLight system, graphic data drawn by a drawing client at a remote location is superimposed and projected on real-world objects such as a whiteboard, poster, and sticky note with a video projector provided, and the image data projected on the real-world objects is captured with the camera provided at the server's end. Then, the image data captured can be displayed on the drawing client provided at the remote location.
In addition, IBM T. J. Watson Research Center proposed the projector camera system as described in Claudio Pinhanez and Gopal Pingali: “Projector-camera Systems for Telepresence”, ETP'04, Oct. 15, 2004 (hereinafter, simply referred to as Document 2). In this system, the projector and camera are used together as in the iLight system, making it possible to project the graphic data drawn at the remote site on the real-word objects on site or on another local site with the projector. Furthermore, as in the iLight system, the graphic data and the real-world objects can be captured with the camera system, and such captured data can be sent to the remote site.
Additionally, there has been proposed a projector having a slant angle measuring device, as described in Japanese Patent Application Publication No. 2005-006272 (hereinafter, referred to as Document 3). The slant angle measuring device measures the slant angles in the vertical and horizontal directions relative to a light axis of the projector so as to correct the image distortion. There has also been proposed another projector, as described in Japanese Patent Application Publication No. 2005-017350 (hereinafter, referred to as Document 4). This projector suppresses the reduction in brightness of visible lights, displays the test pattern with the visible lights on the screen during the normal projection, measures the slant angles in the vertical and horizontal directions relative to the light axis of the projector, and adjusts the focus automatically.
Also, there has been proposed a projection device having a desktop camera and digitizer, as described in Japanese Patent Application Publication No. 2004-265185 (hereinafter, referred to as Document 4). This projection device processes the graphic data of paper data or material for reference as electronic data on the remote meeting or electronic conference.
Document 1, however, in the system that the projection image data projected by the projector is captured with the camera and sent to a remote location, in cases which the projector and camera are structurally or physically separated, there is a problem in that it is difficult to adjust the projector and camera correctly. For example, there are problems in that the relative positions of the image data projected by the projector and the image data captured by the camera are misaligned due to the difference in the light axes of the projector and camera, the difference in the angle of field, or the problem in the mechanical arrangement, the difference in the image size, and the problem caused by the distortion due to scaling up or down or the distortion due to parallel displacement and rotation. If the afore-mentioned problems are not corrected, there will cause another problem that the both of image data cannot be sent or received between the remote sites precisely. Also, the misalignment of the projector and the camera relative to the image data can be adjusted manually from the remote site, yet there is still another problem that it takes time and efforts to adjust the misalignment.
With respect to Document 3 through 5, the techniques described in Document 3 through 5 do not intend to control between the projector and camera.